In [27]:
sns.displot( data = final_lf_b.loc[final_lf_b.Qpass==1], x='sess', col='sid', col_wrap=2,
hue='sess', kind='hist', binwidth = 1, facet_kws={'sharey':False, 'sharex':False})
plt.show()
Educational Process Mining (EPM): A Learning Analytics Data Set. (2015). UCI Machine Learning Repository.
The final exam consisted of 16 questions where each question focused on content from a specific session. Each session has two or more related questions follows:
The questions correspond to trials, and the number of questions answered correctly correspond to events. Student participation (input data) was recorded by activity, exercise, and session. The number of trials and events for each student and session are counts, which can be modeled using Poisson or Negative Binomial Regression. This notebook uses both Poisson and Negative Binomial Regression to model the number of expected correctly answered final questions.
The original values and principle components were used with both Poisson and Negative Binomial Regression methods. Using the AIC as the performance metric, the best model from all four methods was the purely additive model with categorical features sid and actv_grp and numeric features, either the interpolated variables or the principle components: </br>
final_events ~ sid + actv_grp + numeric_features
The fact that the student ID, activity group, and numeric features were important indicates that both the student and the student's behavior as measured by mouse and keyboard activity are required to determine the outcome.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import statsmodels.formula.api as smf
import statsmodels.api as sm
from patsy import dmatrices
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
CMPINF2120_EPM_FUNC_INCL_Over_Lisa.ipynb includes functions used in this notebook.
%run CMPINF2120_EPM_FUNC_INCL_Over_Lisa.ipynb
inputs_final_sqrt_path = 'https://raw.githubusercontent.com/lisaover/CMPINF2120_project/main/tp_sqrt_inputs_final_df.csv'
final_path = 'https://raw.githubusercontent.com/lisaover/CMPINF2120_project/main/final_df.csv'
pts_path = 'https://raw.githubusercontent.com/lisaover/CMPINF2120_project/main/final_points_lookup.csv'
final_sqrt_init = pd.read_csv(inputs_final_sqrt_path)
final_init = pd.read_csv(final_path)
pts_final_lookup = pd.read_csv(pts_path)
final_sqrt_init.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 2444 entries, 0 to 2443 Data columns (total 82 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 sess 2444 non-null int64 1 sid 2444 non-null int64 2 actv_grp 2444 non-null object 3 total_ms_tp000_sqrt 2444 non-null float64 4 mw_tp000_sqrt 2444 non-null float64 5 mwc_tp000_sqrt 2444 non-null float64 6 mcl_tp000_sqrt 2444 non-null float64 7 mcr_tp000_sqrt 2444 non-null float64 8 mm_tp000_sqrt 2444 non-null float64 9 ks_tp000_sqrt 2444 non-null float64 10 total_ms_tp010_sqrt 2444 non-null float64 11 mw_tp010_sqrt 2444 non-null float64 12 mwc_tp010_sqrt 2444 non-null float64 13 mcl_tp010_sqrt 2444 non-null float64 14 mcr_tp010_sqrt 2444 non-null float64 15 mm_tp010_sqrt 2444 non-null float64 16 ks_tp010_sqrt 2444 non-null float64 17 total_ms_tp020_sqrt 2444 non-null float64 18 mw_tp020_sqrt 2444 non-null float64 19 mwc_tp020_sqrt 2444 non-null float64 20 mcl_tp020_sqrt 2444 non-null float64 21 mcr_tp020_sqrt 2444 non-null float64 22 mm_tp020_sqrt 2444 non-null float64 23 ks_tp020_sqrt 2444 non-null float64 24 total_ms_tp030_sqrt 2444 non-null float64 25 mw_tp030_sqrt 2444 non-null float64 26 mwc_tp030_sqrt 2444 non-null float64 27 mcl_tp030_sqrt 2444 non-null float64 28 mcr_tp030_sqrt 2444 non-null float64 29 mm_tp030_sqrt 2444 non-null float64 30 ks_tp030_sqrt 2444 non-null float64 31 total_ms_tp040_sqrt 2444 non-null float64 32 mw_tp040_sqrt 2444 non-null float64 33 mwc_tp040_sqrt 2444 non-null float64 34 mcl_tp040_sqrt 2444 non-null float64 35 mcr_tp040_sqrt 2444 non-null float64 36 mm_tp040_sqrt 2444 non-null float64 37 ks_tp040_sqrt 2444 non-null float64 38 total_ms_tp050_sqrt 2444 non-null float64 39 mw_tp050_sqrt 2444 non-null float64 40 mwc_tp050_sqrt 2444 non-null float64 41 mcl_tp050_sqrt 2444 non-null float64 42 mcr_tp050_sqrt 2444 non-null float64 43 mm_tp050_sqrt 2444 non-null float64 44 ks_tp050_sqrt 2444 non-null float64 45 total_ms_tp060_sqrt 2444 non-null float64 46 mw_tp060_sqrt 2444 non-null float64 47 mwc_tp060_sqrt 2444 non-null float64 48 mcl_tp060_sqrt 2444 non-null float64 49 mcr_tp060_sqrt 2444 non-null float64 50 mm_tp060_sqrt 2444 non-null float64 51 ks_tp060_sqrt 2444 non-null float64 52 total_ms_tp070_sqrt 2444 non-null float64 53 mw_tp070_sqrt 2444 non-null float64 54 mwc_tp070_sqrt 2444 non-null float64 55 mcl_tp070_sqrt 2444 non-null float64 56 mcr_tp070_sqrt 2444 non-null float64 57 mm_tp070_sqrt 2444 non-null float64 58 ks_tp070_sqrt 2444 non-null float64 59 total_ms_tp080_sqrt 2444 non-null float64 60 mw_tp080_sqrt 2444 non-null float64 61 mwc_tp080_sqrt 2444 non-null float64 62 mcl_tp080_sqrt 2444 non-null float64 63 mcr_tp080_sqrt 2444 non-null float64 64 mm_tp080_sqrt 2444 non-null float64 65 ks_tp080_sqrt 2444 non-null float64 66 total_ms_tp090_sqrt 2444 non-null float64 67 mw_tp090_sqrt 2444 non-null float64 68 mwc_tp090_sqrt 2444 non-null float64 69 mcl_tp090_sqrt 2444 non-null float64 70 mcr_tp090_sqrt 2444 non-null float64 71 mm_tp090_sqrt 2444 non-null float64 72 ks_tp090_sqrt 2444 non-null float64 73 total_ms_tp100_sqrt 2444 non-null float64 74 mw_tp100_sqrt 2444 non-null float64 75 mwc_tp100_sqrt 2444 non-null float64 76 mcl_tp100_sqrt 2444 non-null float64 77 mcr_tp100_sqrt 2444 non-null float64 78 mm_tp100_sqrt 2444 non-null float64 79 ks_tp100_sqrt 2444 non-null float64 80 final_events 2444 non-null float64 81 final_trials 2444 non-null float64 dtypes: float64(79), int64(2), object(1) memory usage: 1.5+ MB
final_sqrt_init.isna().sum()
sess 0
sid 0
actv_grp 0
total_ms_tp000_sqrt 0
mw_tp000_sqrt 0
..
mcr_tp100_sqrt 0
mm_tp100_sqrt 0
ks_tp100_sqrt 0
final_events 0
final_trials 0
Length: 82, dtype: int64
final_init.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 62 entries, 0 to 61 Data columns (total 18 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 sid 62 non-null int64 1 Es_1q1 62 non-null float64 2 Es_1q2 62 non-null float64 3 Es_2q1 62 non-null float64 4 Es_2q2 62 non-null float64 5 Es_3q1 62 non-null float64 6 Es_3q2 62 non-null float64 7 Es_3q3 62 non-null float64 8 Es_3q4 62 non-null float64 9 Es_3q5 62 non-null float64 10 Es_4q1 62 non-null float64 11 Es_4q2 62 non-null float64 12 Es_5q1 62 non-null float64 13 Es_5q2 62 non-null float64 14 Es_5q3 62 non-null float64 15 Es_6q1 62 non-null float64 16 Es_6q2 62 non-null float64 17 final_score 62 non-null float64 dtypes: float64(17), int64(1) memory usage: 8.8 KB
final_init.isna().sum()
sid 0 Es_1q1 0 Es_1q2 0 Es_2q1 0 Es_2q2 0 Es_3q1 0 Es_3q2 0 Es_3q3 0 Es_3q4 0 Es_3q5 0 Es_4q1 0 Es_4q2 0 Es_5q1 0 Es_5q2 0 Es_5q3 0 Es_6q1 0 Es_6q2 0 final_score 0 dtype: int64
pts_final_lookup.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 17 entries, 0 to 16 Data columns (total 2 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 question 17 non-null object 1 question_points 17 non-null int64 dtypes: int64(1), object(1) memory usage: 400.0+ bytes
pts_final_lookup.isna().sum()
question 0 question_points 0 dtype: int64
Melt final_init and create a session variable
final_lf = final_init.melt( id_vars=['sid']).\
rename(columns={"variable": "question", "value": "quest_scr"}).\
copy()
final_lf['sess'] = final_lf.question.str.extract('(\d)')
Merge final_lf with pts_init and create a pass/fail variable for each student and question
final_lf_b = pd.merge(final_lf, pts_final_lookup, on='question', how='left')
final_lf_b.head()
| sid | question | quest_scr | sess | question_points | |
|---|---|---|---|---|---|
| 0 | 1 | Es_1q1 | 2.0 | 1 | 2 |
| 1 | 2 | Es_1q1 | 2.0 | 1 | 2 |
| 2 | 4 | Es_1q1 | 2.0 | 1 | 2 |
| 3 | 5 | Es_1q1 | 2.0 | 1 | 2 |
| 4 | 7 | Es_1q1 | 2.0 | 1 | 2 |
final_lf_b['Qpass'] = [1 if i/j >= 0.7 else 0 for (i, j) in zip(final_lf_b['quest_scr'],final_lf_b['question_points'])]
final_lf_b.head()
| sid | question | quest_scr | sess | question_points | Qpass | |
|---|---|---|---|---|---|---|
| 0 | 1 | Es_1q1 | 2.0 | 1 | 2 | 1 |
| 1 | 2 | Es_1q1 | 2.0 | 1 | 2 | 1 |
| 2 | 4 | Es_1q1 | 2.0 | 1 | 2 | 1 |
| 3 | 5 | Es_1q1 | 2.0 | 1 | 2 | 1 |
| 4 | 7 | Es_1q1 | 2.0 | 1 | 2 | 1 |
final_sqrt_init['sid'] = final_sqrt_init['sid'].astype('object')
final_sqrt_init['sess'] = final_sqrt_init['sess'].astype('object')
final_sqrt_df = final_sqrt_init.copy()
sqrt_vars = get_var_list(final_sqrt_df,['sqrt'])
sqrt_features_df = final_sqrt_df.loc[:, sqrt_vars].copy()
sqrt_features_df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 2444 entries, 0 to 2443 Data columns (total 77 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 total_ms_tp000_sqrt 2444 non-null float64 1 mw_tp000_sqrt 2444 non-null float64 2 mwc_tp000_sqrt 2444 non-null float64 3 mcl_tp000_sqrt 2444 non-null float64 4 mcr_tp000_sqrt 2444 non-null float64 5 mm_tp000_sqrt 2444 non-null float64 6 ks_tp000_sqrt 2444 non-null float64 7 total_ms_tp010_sqrt 2444 non-null float64 8 mw_tp010_sqrt 2444 non-null float64 9 mwc_tp010_sqrt 2444 non-null float64 10 mcl_tp010_sqrt 2444 non-null float64 11 mcr_tp010_sqrt 2444 non-null float64 12 mm_tp010_sqrt 2444 non-null float64 13 ks_tp010_sqrt 2444 non-null float64 14 total_ms_tp020_sqrt 2444 non-null float64 15 mw_tp020_sqrt 2444 non-null float64 16 mwc_tp020_sqrt 2444 non-null float64 17 mcl_tp020_sqrt 2444 non-null float64 18 mcr_tp020_sqrt 2444 non-null float64 19 mm_tp020_sqrt 2444 non-null float64 20 ks_tp020_sqrt 2444 non-null float64 21 total_ms_tp030_sqrt 2444 non-null float64 22 mw_tp030_sqrt 2444 non-null float64 23 mwc_tp030_sqrt 2444 non-null float64 24 mcl_tp030_sqrt 2444 non-null float64 25 mcr_tp030_sqrt 2444 non-null float64 26 mm_tp030_sqrt 2444 non-null float64 27 ks_tp030_sqrt 2444 non-null float64 28 total_ms_tp040_sqrt 2444 non-null float64 29 mw_tp040_sqrt 2444 non-null float64 30 mwc_tp040_sqrt 2444 non-null float64 31 mcl_tp040_sqrt 2444 non-null float64 32 mcr_tp040_sqrt 2444 non-null float64 33 mm_tp040_sqrt 2444 non-null float64 34 ks_tp040_sqrt 2444 non-null float64 35 total_ms_tp050_sqrt 2444 non-null float64 36 mw_tp050_sqrt 2444 non-null float64 37 mwc_tp050_sqrt 2444 non-null float64 38 mcl_tp050_sqrt 2444 non-null float64 39 mcr_tp050_sqrt 2444 non-null float64 40 mm_tp050_sqrt 2444 non-null float64 41 ks_tp050_sqrt 2444 non-null float64 42 total_ms_tp060_sqrt 2444 non-null float64 43 mw_tp060_sqrt 2444 non-null float64 44 mwc_tp060_sqrt 2444 non-null float64 45 mcl_tp060_sqrt 2444 non-null float64 46 mcr_tp060_sqrt 2444 non-null float64 47 mm_tp060_sqrt 2444 non-null float64 48 ks_tp060_sqrt 2444 non-null float64 49 total_ms_tp070_sqrt 2444 non-null float64 50 mw_tp070_sqrt 2444 non-null float64 51 mwc_tp070_sqrt 2444 non-null float64 52 mcl_tp070_sqrt 2444 non-null float64 53 mcr_tp070_sqrt 2444 non-null float64 54 mm_tp070_sqrt 2444 non-null float64 55 ks_tp070_sqrt 2444 non-null float64 56 total_ms_tp080_sqrt 2444 non-null float64 57 mw_tp080_sqrt 2444 non-null float64 58 mwc_tp080_sqrt 2444 non-null float64 59 mcl_tp080_sqrt 2444 non-null float64 60 mcr_tp080_sqrt 2444 non-null float64 61 mm_tp080_sqrt 2444 non-null float64 62 ks_tp080_sqrt 2444 non-null float64 63 total_ms_tp090_sqrt 2444 non-null float64 64 mw_tp090_sqrt 2444 non-null float64 65 mwc_tp090_sqrt 2444 non-null float64 66 mcl_tp090_sqrt 2444 non-null float64 67 mcr_tp090_sqrt 2444 non-null float64 68 mm_tp090_sqrt 2444 non-null float64 69 ks_tp090_sqrt 2444 non-null float64 70 total_ms_tp100_sqrt 2444 non-null float64 71 mw_tp100_sqrt 2444 non-null float64 72 mwc_tp100_sqrt 2444 non-null float64 73 mcl_tp100_sqrt 2444 non-null float64 74 mcr_tp100_sqrt 2444 non-null float64 75 mm_tp100_sqrt 2444 non-null float64 76 ks_tp100_sqrt 2444 non-null float64 dtypes: float64(77) memory usage: 1.4 MB
sqrt_feature_names = sqrt_features_df.columns
len(sqrt_feature_names)
77
final_sqrt_df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 2444 entries, 0 to 2443 Data columns (total 82 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 sess 2444 non-null object 1 sid 2444 non-null object 2 actv_grp 2444 non-null object 3 total_ms_tp000_sqrt 2444 non-null float64 4 mw_tp000_sqrt 2444 non-null float64 5 mwc_tp000_sqrt 2444 non-null float64 6 mcl_tp000_sqrt 2444 non-null float64 7 mcr_tp000_sqrt 2444 non-null float64 8 mm_tp000_sqrt 2444 non-null float64 9 ks_tp000_sqrt 2444 non-null float64 10 total_ms_tp010_sqrt 2444 non-null float64 11 mw_tp010_sqrt 2444 non-null float64 12 mwc_tp010_sqrt 2444 non-null float64 13 mcl_tp010_sqrt 2444 non-null float64 14 mcr_tp010_sqrt 2444 non-null float64 15 mm_tp010_sqrt 2444 non-null float64 16 ks_tp010_sqrt 2444 non-null float64 17 total_ms_tp020_sqrt 2444 non-null float64 18 mw_tp020_sqrt 2444 non-null float64 19 mwc_tp020_sqrt 2444 non-null float64 20 mcl_tp020_sqrt 2444 non-null float64 21 mcr_tp020_sqrt 2444 non-null float64 22 mm_tp020_sqrt 2444 non-null float64 23 ks_tp020_sqrt 2444 non-null float64 24 total_ms_tp030_sqrt 2444 non-null float64 25 mw_tp030_sqrt 2444 non-null float64 26 mwc_tp030_sqrt 2444 non-null float64 27 mcl_tp030_sqrt 2444 non-null float64 28 mcr_tp030_sqrt 2444 non-null float64 29 mm_tp030_sqrt 2444 non-null float64 30 ks_tp030_sqrt 2444 non-null float64 31 total_ms_tp040_sqrt 2444 non-null float64 32 mw_tp040_sqrt 2444 non-null float64 33 mwc_tp040_sqrt 2444 non-null float64 34 mcl_tp040_sqrt 2444 non-null float64 35 mcr_tp040_sqrt 2444 non-null float64 36 mm_tp040_sqrt 2444 non-null float64 37 ks_tp040_sqrt 2444 non-null float64 38 total_ms_tp050_sqrt 2444 non-null float64 39 mw_tp050_sqrt 2444 non-null float64 40 mwc_tp050_sqrt 2444 non-null float64 41 mcl_tp050_sqrt 2444 non-null float64 42 mcr_tp050_sqrt 2444 non-null float64 43 mm_tp050_sqrt 2444 non-null float64 44 ks_tp050_sqrt 2444 non-null float64 45 total_ms_tp060_sqrt 2444 non-null float64 46 mw_tp060_sqrt 2444 non-null float64 47 mwc_tp060_sqrt 2444 non-null float64 48 mcl_tp060_sqrt 2444 non-null float64 49 mcr_tp060_sqrt 2444 non-null float64 50 mm_tp060_sqrt 2444 non-null float64 51 ks_tp060_sqrt 2444 non-null float64 52 total_ms_tp070_sqrt 2444 non-null float64 53 mw_tp070_sqrt 2444 non-null float64 54 mwc_tp070_sqrt 2444 non-null float64 55 mcl_tp070_sqrt 2444 non-null float64 56 mcr_tp070_sqrt 2444 non-null float64 57 mm_tp070_sqrt 2444 non-null float64 58 ks_tp070_sqrt 2444 non-null float64 59 total_ms_tp080_sqrt 2444 non-null float64 60 mw_tp080_sqrt 2444 non-null float64 61 mwc_tp080_sqrt 2444 non-null float64 62 mcl_tp080_sqrt 2444 non-null float64 63 mcr_tp080_sqrt 2444 non-null float64 64 mm_tp080_sqrt 2444 non-null float64 65 ks_tp080_sqrt 2444 non-null float64 66 total_ms_tp090_sqrt 2444 non-null float64 67 mw_tp090_sqrt 2444 non-null float64 68 mwc_tp090_sqrt 2444 non-null float64 69 mcl_tp090_sqrt 2444 non-null float64 70 mcr_tp090_sqrt 2444 non-null float64 71 mm_tp090_sqrt 2444 non-null float64 72 ks_tp090_sqrt 2444 non-null float64 73 total_ms_tp100_sqrt 2444 non-null float64 74 mw_tp100_sqrt 2444 non-null float64 75 mwc_tp100_sqrt 2444 non-null float64 76 mcl_tp100_sqrt 2444 non-null float64 77 mcr_tp100_sqrt 2444 non-null float64 78 mm_tp100_sqrt 2444 non-null float64 79 ks_tp100_sqrt 2444 non-null float64 80 final_events 2444 non-null float64 81 final_trials 2444 non-null float64 dtypes: float64(79), object(3) memory usage: 1.5+ MB
sns.displot( data = final_lf_b.loc[final_lf_b.Qpass==1], x='sess', col='sid', col_wrap=2,
hue='sess', kind='hist', binwidth = 1, facet_kws={'sharey':False, 'sharex':False})
plt.show()
final_sqrt_lf = final_sqrt_df.melt(id_vars=['sess','sid','actv_grp','final_events','final_trials'], ignore_index=True).copy()
final_sqrt_lf.head()
| sess | sid | actv_grp | final_events | final_trials | variable | value | |
|---|---|---|---|---|---|---|---|
| 0 | 1 | 1 | Aulaweb | 2.0 | 2.0 | total_ms_tp000_sqrt | 89.442719 |
| 1 | 1 | 1 | Blank | 2.0 | 2.0 | total_ms_tp000_sqrt | 89.442719 |
| 2 | 1 | 1 | Deeds | 2.0 | 2.0 | total_ms_tp000_sqrt | 202.484567 |
| 3 | 1 | 1 | Diagram | 2.0 | 2.0 | total_ms_tp000_sqrt | 939.148551 |
| 4 | 1 | 1 | Other | 2.0 | 2.0 | total_ms_tp000_sqrt | 31.622777 |
final_sqrt_lf.actv_grp.unique()
array(['Aulaweb', 'Blank', 'Deeds', 'Diagram', 'Other', 'Properties',
'Study', 'TextEditor', 'Study_Materials', 'FSM_Related', 'FSM'],
dtype=object)
actv_subgrp_1 = ['Aulaweb','Deeds','Diagram','TextEditor','FSM_Related','FSM']
actv_subgrp_2 = ['Blank','Other','Properties','Study','Study_Materials']
sns.relplot( data = final_sqrt_lf.loc[final_sqrt_lf.actv_grp.isin(actv_subgrp_1) & (final_sqrt_lf['sess']==1)],
x='final_events', y='value',
col='actv_grp', row='variable', hue='actv_grp',
facet_kws={'sharey': False, 'sharex': False})
plt.show()
sns.relplot( data = final_sqrt_lf.loc[final_sqrt_lf.actv_grp.isin(actv_subgrp_1) & (final_sqrt_lf['sess']==2)],
x='final_events', y='value',
col='actv_grp', row='variable', hue='actv_grp',
facet_kws={'sharey': False, 'sharex': False})
plt.show()
